Efficient and Robust Point Cloud Registration via Heuristics-guided Parameter Search.

Estimating the rigid transformation with 6 degrees of freedom based on a putative 3D correspondence set is a crucial procedure in point cloud registration. Existing correspondence identification methods usually lead to large outlier ratios (> 95% is common), underscoring the significance of robust registration methods. Many researchers turn to parameter search-based strategies (e.g., Branch-and-Bround) for robust registration. Although related methods show high robustness, their efficiency is limited to the high-dimensional search space. This paper proposes a heuristics-guided parameter search strategy to accelerate the search while maintaining high robustness. We first sample some correspondences (i.e., heuristics) and then just need to sequentially search the feasible regions that make each sample an inlier. Our strategy largely reduces the search space and can guarantee accuracy with only a few inlier samples, therefore enjoying an excellent trade-off between efficiency and robustness. Since directly parameterizing the 6-dimensional nonlinear feasible region for efficient search is intractable, we construct a three-stage decomposition pipeline to reparameterize the feasible region, resulting in three lower-dimensional sub-problems that are easily solvable via our strategy. Besides reducing the searching dimension, our decomposition enables the leverage of 1-dimensional interval stabbing at all three stages for searching acceleration. Moreover, we propose a valid sampling strategy to guarantee our sampling effectiveness, and a compatibility verification setup to further accelerate our search. Extensive experiments on both simulated and real-world datasets demonstrate that our approach exhibits comparable robustness with state-of-the-art methods while achieving a significant efficiency boost.

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments.

In this article, we present a novel and generic data-driven method to servo-control the 3-D shape of continuum and soft robots based on proprioceptive sensing feedback. Developments of 3-D shape perception and control technologies are crucial for continuum and soft robots to perform tasks autonomously in surgical interventions. However, owing to the nonlinear properties of continuum robots, one main difficulty lies in the modeling of them, especially for soft robots with variable stiffness. To address this problem, we propose a versatile learning-based adaptive shape controller by leveraging proprioception of 3-D configuration from fiber Bragg grating (FBG) sensors, which can online estimate the unknown model of continuum robot against unexpected disturbances and exhibit an adaptive behavior to the unmodeled system without priori data exploration. Based on a new composite adaptation algorithm, the asymptotic convergences of the closed-loop system with learning parameters have been proven by Lyapunov theory. To validate the proposed method, we present a comprehensive experimental study using two continuum and soft robots both integrated with multicore FBGs, including a robotic-assisted colonoscope and multisection extensible soft manipulators. The results demonstrate the feasibility, adaptability, and superiority of our controller in various unstructured environments, as well as phantom experiments.

C-C chemokine receptor 5 is essential for conventional NK cell trafficking and liver injury in a murine hepatitis virus-induced fulminant hepatic failure model.

BACKGROUND: Previous studies have demonstrated that natural killer (NK) cells migrated into the liver from peripheral organs and exerted cytotoxic effects on hepatocytes in virus-induced liver failure. AIM: This study aimed to investigate the potential therapeutic role of chemokine receptors in the migration of NK cells in a murine hepatitis  virus strain 3 (MHV-3)-induced fulminant hepatic failure (MHV-3-FHF) model and its mechanism. RESULTS: By gene array analysis, chemokine (C-C motif) receptor 5 (CCR5) was found to have remarkably elevated expression levels in hepatic NK cells after MHV-3 infection. The number of hepatic CCR5+ conventional NK (cNK) cells increased and peaked at 48 h after MHV-3 infection, while the number of hepatic resident NK (rNK) cells steadily declined. Moreover, the expression of CCR5-related chemokines, including macrophage inflammatory protein (MIP)-1α, MIP-1ß and regulated on activation, normal T-cell expressed and secreted (RANTES) was significantly upregulated in MHV-3-infected hepatocytes. In an in vitro Transwell migration assay, CCR5-blocked splenic cNK cells showed decreased migration towards MHV-3-infected hepatocytes, and inhibition of MIP-1ß or RANTES but not MIP-1α decreased cNK cell migration. Moreover, CCR5 knockout (KO) mice displayed reduced infiltration of hepatic cNK cells after MHV-3 infection, accompanied by attenuated liver injury and improved mouse survival time. Adoptive transfer of cNK cells from wild-type mice into CCR5 KO mice resulted in the abundant accumulation of hepatic cNK cells and aggravated liver injury. Moreover, pharmacological inhibition of CCR5 by maraviroc reduced cNK cell infiltration in the liver and liver injury in the MHV-3-FHF model. CONCLUSION: The CCR5-MIP-1ß/RANTES axis played a critical role in the recruitment of cNK cells to the liver during MHV-3-induced liver injury. Targeted inhibition of CCR5 provides a therapeutic approach to ameliorate liver damage during virus-induced acute liver injury.

Hong Kong World: Leveraging Structural Regularity for Line-Based SLAM.

Manhattan and Atlanta worlds hold for the structured scenes with only vertical and horizontal dominant directions (DDs). To describe the scenes with additional sloping DDs, a mixture of independent Manhattan worlds seems plausible, but may lead to unaligned and unrelated DDs. By contrast, we propose a novel structural model called Hong Kong world. It is more general than Manhattan and Atlanta worlds since it can represent the environments with slopes, e.g., a city with hilly terrain, a house with sloping roof, and a loft apartment with staircase. Moreover, it is more compact and accurate than a mixture of independent Manhattan worlds by enforcing the orthogonality constraints between not only vertical and horizontal DDs, but also horizontal and sloping DDs. We further leverage the structural regularity of Hong Kong world for the line-based SLAM. Our SLAM method is reliable thanks to three technical novelties. First, we estimate DDs/vanishing points in Hong Kong world in a semi-searching way. We use a new consensus voting strategy for search, instead of traditional branch and bound. This method is the first one that can simultaneously determine the number of DDs, and achieve quasi-global optimality in terms of the number of inliers. Second, we compute the camera pose by exploiting the spatial relations between DDs in Hong Kong world. This method generates concise polynomials, and thus is more accurate and efficient than existing approaches designed for unstructured scenes. Third, we refine the estimated DDs in Hong Kong world by a novel filter-based method. Then we use these refined DDs to optimize the camera poses and 3D lines, leading to higher accuracy and robustness than existing optimization algorithms. In addition, we establish the first dataset of sequential images in Hong Kong world. Experiments showed that our approach outperforms state-of-the-art methods in terms of accuracy and/or efficiency.

Design, Control, and Experiments of a Novel Robotic Uterine Manipulator With the Motorized 3-DoF Manipulation Rod.

To address the issue of declining performance over time with manual uterine manipulation during minimally invasive gynecologic surgery, we propose a novel uterine manipulation robot that consists of a 3-DoF remote center of motion (RCM) mechanism and a 3-DoF manipulation rod. This allows for tireless, stable, and safer manipulation in place of a human assistant. For the RCM mechanism, we propose a single-motor bilinear-guided mechanism that can achieve a wide range of pitch motion (-50  âˆ¼  34 degrees) while maintaining a compact structure. This novel uterine manipulation robot is equipped with a manipulation rod that has a tip diameter of only 6 mm, allowing it to accommodate almost any patient's cervix. The 30-degree distal pitch motion and ±45-degree distal roll motion of the instrument further improve uterine visualization. Additionally, the tip of the rod can be opened into a T-shape to minimize damage to the uterus. Laboratory experiments have shown the mechanical RCM accuracy of 0.373 mm and the maximum load of the distal pitch joint of 500 g. Feasibility has been demonstrated through ex-vivo and cadaver tests, as well as clinical trials.

Development of an Open Microfluidic Platform for Oocyte One-Stop Vitrification with Cryotop Method.

Oocyte vitrification technology is widely used for assisted reproduction and fertility preservation, which requires precise washing sequences and timings of cryoprotectant agents (CPAs) treatment to relieve the osmotic shock to cells. The gold standard Cryotop method is extensively used in oocyte vitrification and is currently the most commonly used method in reproductive centers. However, the Cryotop method requires precise and complex manual manipulation by an embryologist, whose proficiency directly determines the effect of vitrification. Therefore, in this study, an automatic microfluidic system consisting of a novel open microfluidic chip and a set of automatic devices was established as a standardized operating protocol to facilitate the conventional manual Cryotop method and minimize the osmotic shock applied to the oocyte. The proposed open microfluidic system could smoothly change the CPA concentration around the oocyte during vitrification pretreatment, and transferred the treated oocyte to the Cryotop with a tiny droplet. The system better conformed to the operating habits of embryologists, whereas the integration of commercialized Cryotop facilitates the subsequent freezing and thawing processes. With standardized operating procedures, our system provides consistent treatment effects for each operation, leading to comparable survival rate, mitochondrial membrane potential (MMP) and reactive oxygen species (ROS) level of oocytes to the manual Cryotop operations. The vitrification platform is the first reported microfluidic system integrating the function of cells transfer from the processing chip, which avoids the risk of cell loss or damage in a manual operation and ensures the sufficient cooling rate during liquid nitrogen (LN2) freezing. Our study demonstrates significant potential of the automatic microfluidic approach to serve as a facile and universal solution for the vitrification of various precious cells.

A Robotic System With Embedded Open Microfluidic Chip for Automatic Embryo Vitrification.

Embryo vitrification is a fundamental technology utilized in assisted reproduction and fertility preservation. Vitrification involves sequential loading and unloading of cryoprotectants (CPAs) with strict time control, and transferring the embryo in a minimum CPA droplet to the vitrification straw. However, manual operation still cannot effectively avoid embryo loss, and the existing automatic vitrification systems have insufficient system reliability, and operate differently from clinical vitrification protocol. Through collaboration with in vitro fertilization (IVF) clinics, we are in the process realizing a robotic system that can automatically conduct the embryo vitrification process, including the pretreatment with CPAs, transfer of embryo to the vitrification straw, and cryopreservation with liquid nitrogen ( LN2). An open microfluidic chip (OMC) was designed to accommodate the embryo during the automatic CPAs pretreatment process. The design of two chambers connected by a capillary gap facilitated solution exchange around the embryo, and simultaneously reduced the risk of embryo loss in the flow field. In accordance to the well-accepted procedure and medical devices in manual operation, we designed the entire vitrification protocol, as well as the robotic prototype. In a practical experiment using mouse embryos, our robotic system showed a 100 % success rate in transferring and vitrifying the embryos, achieved comparable embryo survival rates (90.9 % versus 94.4 %) and development rates (90.0 % versus 94.1 %), when compared with the manual group conducted by the senior embryologist. With this study, we aim to facilitate the standardization of clinical vitrification from manual operation to a more efficient and reliable automated process.

3D surface reconstruction of transparent objects using laser scanning with a four-layers refinement process.

Acquiring the 3D geometry of objects has been an active research topic, wherein the reconstruction of transparent objects poses a great challenge. In this paper, we present a fully automatic approach for reconstructing the exterior surface of a complex transparent scene. Through scanning a line laser by a galvo-mirror, images of the scene are captured from two viewing directions. Due to the light transmission inside the transparent object, the captured feature points and the calibrated laser plane can produce large number of 3D point candidates with large incorrect points through direct triangulation. Various situations of laser transmission inside the transparent object are analyzed and the reconstructed 3D laser point candidates are classified into two types: first-reflection points and non-first-reflection points. The first-reflection points means the first reflected laser points on the front surface of measured objects. Then, a novel four-layers refinement process is proposed to extract the first-reflection points step by step from the 3D point candidates through optical geometric constraints, including (1) Layer-1 : fake points removed by single camera, (2) Layer-2 : ambiguity points removed by the dual-camera joint constraint, (3) Layer-3 : retrieve the missing first-reflection exterior surface points by fusion and (4) Layer-4 : severe ambiguity points removed by contour-continuity. Besides, a novel calibration model about this imaging system is proposed for 3D point candidates reconstruction through triangulation. Compared with traditional laser scanning method, we pulled in the viewing angle information of the second camera and a novel four-layers refinement process is adopted for reconstruction of transparent objects. Various experiments on real objects demonstrate that proposed method can successfully extract the first-reflection points from the candidates and recover the complex shapes of transparent and semitransparent objects.

Cav3.1-driven bursting firing in ventromedial hypothalamic neurons exerts dual control of anxiety-like behavior and energy expenditure.

The central nervous system has evolved to coordinate the regulation of both the behavior response to the external environment and homeostasis of energy expenditure. Recent studies have indicated the dorsomedial ventromedial hypothalamus (dmVMH) as an important hub that regulates both innate behavior and energy homeostasis for coping stress. However, how dmVMH neurons control neuronal firing pattern to regulate chronic stress-induced anxiety and energy expenditure remains poorly understood. Here, we found enhanced neuronal activity in VMH after chronic stress, which is mainly induced by increased proportion of burst firing neurons. This enhancement of VMH burst firing is predominantly mediated by Cav3.1 expression. Optogenetically evoked burst firing of dmVMH neurons induced anxiety-like behavior, shifted the respiratory exchange ratio toward fat oxidation, and decreased food intake, while knockdown of Cav3.1 in the dmVMH had the opposite effects, suggested that Cav 3.1 as a crucial regulator. Interestingly, we found that fluoxetine (anxiolytics) could block the increase of Cav3.1 expression to inhibit the burst firing, and then rescued the anxiety-like behaviors and energy expenditure changes. Collectively, our study first revealed an important role of Cav3.1-driven bursting firing of dmVMH neurons in the control of anxiety-like behavior and energy expenditure, and provided potential therapeutic targets for treating the chronic stress-induced emotional malfunction and metabolism disorders.

[Effects of landscape complexity and local management on bee pollinator diversity in apple orchards in Changping District, Beijing, China.] / åŒ—äº¬æ˜Œå¹³åŒºæ™¯è§‚å¤æ‚åº¦å’Œå±€åœ°ç®¡ç†å¯¹è‹¹æžœå›­ä¼ ç²‰èœ‚å¤šæ ·æ€§çš„å½±å“.

Pollinators provide important ecosystem services for crop production and food security. With the development of agricultural economy and the increasing intensity of land-use, a large number of natural or semi-natural habitats have been converted to croplands. Landscape homogenization and intensive management lead to the decline of wild bee diversity and threaten the sustainable agricultural production. In this study, we investigated the effects of landscape complexity (proportion of semi-natural habitats), local management practices (local flowering plant diversity and soil total nitrogen), and their interactions on diversity of bee pollinators in apple orchard in Changping District, Beijing. A total of 8642 bee individuals were captured, including 5125 honey bees and 3517 wild bees from 5 families, 14 genera, and 49 species. The optimal landscape scale for the response of bee diversity to landscape complexity and local management intensity was 500 m. Within 500 m radius of the site, the abundance of overall bees and wild bees significantly increased with increasing proportion of semi-natural habitats. The landscape complexity interacting with local flowering plant diversity significantly affected the richness of overall bee and wild bee. When the proportion of semi-natural habitats surrounding the apple orchards was low (≤29.9%), we found a positive effect of flowering plant diversity on the richness of overall bee and wild bee, whereas a reversed trend was found when the proportion of semi-natural habitats surrounding the apple orchards was high (>29.9%). In addition, the abundance of honey bees significantly increased with the increase of local flowering plant diversity and soil total nitrogen. The soil total nitrogen interacting with local flowering plant diversity significantly affected the honey bee abundance. At low levels of soil total nitrogen (≤1.9 g·kg-1), there was a positive effect of flowering plant diversity on honey bee abundance; whereas this trend was reversed at high levels of soil total nitrogen (>1.9 g·kg-1). Increasing the proportion of semi-natural habitats in agricultural landscape was beneficial to the increase of wild bee abundance, and flowering plant diversity could promote bee diversity but depending on landscape scale (proportion of semi-natural habitats) and local scale (nitrogen application). Therefore, multi-scale factors should be considered to develop conservation strategies to maintain the diversity of wild bees in agricultural landscape. Maintaining a higher proportion of cultivated land as much as possible is still a long-term requirement for production, while maintaining intermediate landscape complexity, increasing the diversity of flowering plants on the ground, and reducing the application of nitrogen fertilizer would be effective ways to promote the diversity of pollinating bees in apple orchards.

Accurate 3D Reconstruction of Dynamic Objects by Spatial-Temporal Multiplexing and Motion-Induced Error Elimination.

Three-dimensional (3D) reconstruction of dynamic objects has broad applications, including object recognition and robotic manipulation. However, achieving high-accuracy reconstruction and robustness to motion simultaneously is a challenging task. In this paper, we present a novel method for 3D reconstruction of dynamic objectS, whose main features are as follows. Firstly, a structured-light multiplexing method is developed that only requires 3 patterns to achieve high-accuracy encoding. Fewer projected patterns require shorter image acquisition time, thus, the object motion is reduced in each reconstruction cycle. The three patterns, i.e. spatial-temporally encoded patterns, are generated by embedding a specifically designed spatial-coded texture map into the temporal-encoded three-step phase-shifting fringes. A temporal codeword and three spatial codewords are extracted from the composite patterns using a proposed extraction algorithm. The two types of codewords are utilized separately in stereo matching: the temporal codeword ensures the high accuracy, while the spatial codewords are responsible for removing phase ambiguity. Secondly, we aim to eliminate the reconstruction error induced by motion between frames abbreviated as motion induced error (MiE). Instead of assuming the object to be static when acquiring the 3 images, we derive the motion of projection pixels among frames. Using the extracted spatial codewords, correspondences between different frames are found, i.e. pixels with the same codewords are traceable in the image sequences. Therefore, we can obtain the phase map at each image-acquisition moment without being affected by the object motion. Then the object surfaces corresponding to all the images can be recovered. Experimental results validate the high reconstruction accuracy and precision of the proposed method for dynamic objects with different motion speeds. Comparative experiments show that the presented method demonstrates superior performance with various types of motion, including translation in different directions and deformation.

Quasi-Globally Optimal and Near/True Real-Time Vanishing Point Estimation in Manhattan World.

Image lines projected from parallel 3D lines intersect at a common point called the vanishing point (VP). Manhattan world holds for the scenes with three orthogonal VPs. In Manhattan world, given several lines in a calibrated image, we aim to cluster them by three unknown-but-sought VPs. The VP estimation can be reformulated as computing the rotation between the Manhattan frame and camera frame. To estimate three degrees of freedom (DOF) of this rotation, state-of-the-art methods are based on either data sampling or parameter search. However, they fail to guarantee high accuracy and efficiency simultaneously. In contrast, we propose a set of approaches that hybridize these two strategies. We first constrain two or one DOF of the rotation by two or one sampled image line. Then we search for the remaining one or two DOF based on branch and bound. Our sampling accelerates our search by reducing the search space and simplifying the bound computation. Our search achieves quasi-global optimality. Specifically, it guarantees to retrieve the maximum number of inliers on the condition that two or one DOF is constrained. Our hybridization of two-line sampling and one-DOF search can estimate VPs in real time. Our hybridization of one-line sampling and two-DOF search can estimate VPs in near real time. Experiments on both synthetic and real-world datasets demonstrated that our approaches outperform state-of-the-art methods in terms of accuracy and/or efficiency.

Fully Uncalibrated Image-Based Visual Servoing of 2DOFs Planar Manipulators With a Fixed Camera.

We consider the uncalibrated vision-based control problem of robotic manipulators in this work. Though lots of approaches have been proposed to solve this problem, they usually require calibration (offline or online) of the camera parameters in the implementation, and the control performance may be largely affected by parameter estimation errors. In this work, we present new fully uncalibrated visual servoing approaches for position control of the 2DOFs planar manipulator with a fixed camera. In the proposed approaches, no camera calibration is required, and numerical optimization algorithms or adaptive laws for parameter estimation are not needed. One benefit of such features is that exponential convergence of the image position errors can be ensured regardless of the camera parameter uncertainties. Generally, existing uncalibrated approaches only can guarantee asymptotical convergence of the position errors. Moreover, different from most existing approaches which assume that the robot motion plane and the image plane are parallel, one of the proposed approaches allows the camera to be installed at a general pose. This also simplifies the controller implementation and improves the system design flexibility. Finally, simulation and experimental results are provided to illustrate the effectiveness of the presented fully uncalibrated visual servoing approaches.

Self-Supervised Video Representation Learning by Uncovering Spatio-Temporal Statistics.

This paper proposes a novel pretext task to address the self-supervised video representation learning problem. Specifically, given an unlabeled video clip, we compute a series of spatio-temporal statistical summaries, such as the spatial location and dominant direction of the largest motion, the spatial location and dominant color of the largest color diversity along the temporal axis, etc. Then a neural network is built and trained to yield the statistical summaries given the video frames as inputs. In order to alleviate the learning difficulty, we employ several spatial partitioning patterns to encode rough spatial locations instead of exact spatial Cartesian coordinates. Our approach is inspired by the observation that human visual system is sensitive to rapidly changing contents in the visual field, and only needs impressions about rough spatial locations to understand the visual contents. To validate the effectiveness of the proposed approach, we conduct extensive experiments with four 3D backbone networks, i.e., C3D, 3D-ResNet, R(2+1)D and S3D-G. The results show that our approach outperforms the existing approaches across these backbone networks on four downstream video analysis tasks including action recognition, video retrieval, dynamic scene recognition, and action similarity labeling. The source code is publicly available at: https://github.com/laura-wang/video_repres_sts.

Anchor-guided online meta adaptation for fast one-Shot instrument segmentation from robotic surgical videos.

The scarcity of annotated surgical data in robot-assisted surgery (RAS) motivates prior works to borrow related domain knowledge to achieve promising segmentation results in surgical images by adaptation. For dense instrument tracking in a robotic surgical video, collecting one initial scene to specify target instruments (or parts of tools) is desirable and feasible during the preoperative preparation. In this paper, we study the challenging one-shot instrument segmentation for robotic surgical videos, in which only the first frame mask of each video is provided at test time, such that the pre-trained model (learned from easily accessible source) can adapt to the target instruments. Straightforward methods transfer the domain knowledge by fine-tuning the model on each given mask. Such one-shot optimization takes hundred of iterations and the test runtime is unfeasible. We present anchor-guided online meta adaptation (AOMA) for this problem. We achieve fast one-shot test time optimization by meta-learning a good model initialization and learning rates from source videos to avoid the laborious and handcrafted fine-tuning. The trainable two components are optimized in a video-specific task space with a matching-aware loss. Furthermore, we design an anchor-guided online adaptation to tackle the performance drop throughout a robotic surgical sequence. The model is continuously adapted on motion-insensitive pseudo-masks supported by anchor matching. AOMA achieves state-of-the-art results on two practical scenarios: (1) general videos to surgical videos, (2) public surgical videos to in-house surgical videos, while reducing the test runtime substantially.

Soft robotic manipulator for intraoperative MRI-guided transoral laser microsurgery.

Magnetic resonance (MR) imaging (MRI) provides compelling features for the guidance of interventional procedures, including high-contrast soft tissue imaging, detailed visualization of physiological changes, and thermometry. Laser-based tumor ablation stands to benefit greatly from MRI guidance because 3D resection margins alongside thermal distributions can be evaluated in real time to protect critical structures while ensuring adequate resection margins. However, few studies have investigated the use of projection-based lasers like those for transoral laser microsurgery, potentially because dexterous laser steering is required at the ablation site, raising substantial challenges in the confined MRI bore and its strong magnetic field. Here, we propose an MR-safe soft robotic system for MRI-guided transoral laser microsurgery. Owing to its miniature size (Ø12 × 100 mm), inherent compliance, and five degrees of freedom, the soft robot ensures zero electromagnetic interference with MRI and enables safe and dexterous operation within the confined oral and pharyngeal cavities. The laser manipulator is rapidly fabricated with hybrid soft and hard structures and is powered by microvolume (<0.004 milliter) fluid flow to enable laser steering with enhanced stiffness and lowered hysteresis. A learning-based controller accommodates the inherent nonlinear robot actuation, which was validated with laser path-following tests. Submillimeter laser steering accuracy was demonstrated with a mean error < 0.20 mm. MRI compatibility testing demonstrated zero observable image artifacts during robot operation. Ex vivo tissue ablation and a cadaveric head-and-neck trial were carried out under MRI, where we employed MR thermometry to monitor the tissue ablation margin and thermal diffusion intraoperatively.

Accurate instance segmentation of surgical instruments in robotic surgery: model refinement and cross-dataset evaluation.

PURPOSE: Automatic segmentation of surgical instruments in robot-assisted minimally invasive surgery plays a fundamental role in improving context awareness. In this work, we present an instance segmentation model based on refined Mask R-CNN for accurately segmenting the instruments as well as identifying their types. METHODS: We re-formulate the instrument segmentation task as an instance segmentation task. Then we optimize the Mask R-CNN with anchor optimization and improved Region Proposal Network for instrument segmentation. Moreover, we perform cross-dataset evaluation with different sampling strategies. RESULTS: We evaluate our model on a public dataset of the MICCAI 2017 Endoscopic Vision Challenge with two segmentation tasks, and both achieve new state-of-the-art performance. Besides, cross-dataset training improved the performance on both segmentation tasks compared with those tested on the public dataset. CONCLUSION: Results demonstrate the effectiveness of the proposed instance segmentation network for surgical instruments segmentation. Cross-dataset evaluation shows our instance segmentation model presents certain cross-dataset generalization capability, and cross-dataset training can significantly improve the segmentation performance. Our empirical study also provides guidance on how to allocate the annotation cost for surgeons while labelling a new dataset in practice.

Honeycomb Jamming: An Enabling Technology of Variable Stiffness Reconfiguration.

Jamming technologies are one of the promising approaches of variable stiffness mechanisms. However, there are problems limiting the broad application of jamming-based approaches such as a limited stiffening capacity and restricted stiffening position. This article presents a variable stiffness mechanism to achieve a rapid flexible to rigid state transition with biocompatibility, fail-safe design, and enhanced stiffening capacity. A novel strategy of reconfiguration of stiffening regions, which is entitled variable stiffness reconfiguration, is exploited to control not only the stiffnesses but also the positions and areas of the stiffening regions. At first, this article provides a new approach to the variable stiffness soft robotics community to enable both stiffness control and stiffening region adjustment. In this way, additional functions of the variable stiffness mechanisms including reproducing complex manipulator postures or customizing the soft gripper, through delivering functional units into or out of the devices, are demonstrated. Through reconfiguration, our design provides a generally applicable solution for a wide range of complex manipulator postures reproduced and objects grasped by reconfiguration of the stiffening regions. The variable stiffness mechanism is empirically evaluated with a comparison with other variable stiffness strategies in which the proposed solution shows greater stiffening capability, and an experimental search of optimal parameters of the honeycomb structure is presented. Finite element models, which have shown reasonable agreement with the empirical results, are constructed to model the stiffnesses, and an analytic model of the manipulator is derived to predict the posture.

Consensus With Persistently Exciting Couplings and Its Application to Vision-Based Estimation.

The problem of consensus in networked agent systems is revisited and applied to vision-based localization. A class of new consensus dynamics is introduced first, and sufficient conditions including the persistence of excitation on the coupling matrix for reaching consensus are derived. As an application of the proposed consensus dynamics, an adaptive localization algorithm then is proposed for autonomous robots equipped with primarily visual sensors in GPS-denied environments. In the context of consensus over an undirected tree topology, the convergence of the proposed localization algorithm is proved. Finally, both numerical simulations and physical experiments are presented to show the effectiveness of the proposed localization algorithm. Our algorithm is simpler to implement and computationally cheaper compared to other localization methods. Moreover, it is immune to error accumulation and long-term stable, and the asymptotical convergence of the estimation errors can be theoretically guaranteed.

Tele-Operated Oropharyngeal Swab (TOOS) Robot Enabled by TSS Soft Hand for Safe and Effective Sampling.

The COVID-19 pandemic has imposed serious challenges in multiple perspectives of human life. To diagnose COVID-19, oropharyngeal swab (OP SWAB) sampling is generally applied for viral nucleic acid (VNA) specimen collection. However, manual sampling exposes medical staff to a high risk of infection. Robotic sampling is promising to mitigate this risk to the minimum level, but traditional robot suffers from safety, cost, and control complexity issues for wide-scale deployment. In this work, we present soft robotic technology is promising to achieve robotic OP swab sampling with excellent swab manipulability in a confined oral space and works as dexterous as existing manual approach. This is enabled by a novel Tstone soft (TSS) hand, consisting of a soft wrist and a soft gripper, designed from human sampling observation and bio-inspiration. TSS hand is in a compact size, exerts larger workspace, and achieves comparable dexterity compared to human hand. The soft wrist is capable of agile omnidirectional bending with adjustable stiffness. The terminal soft gripper is effective for disposable swab pinch and replacement. The OP sampling force is easy to be maintained in a safe and comfortable range (throat sampling comfortable region) under a hybrid motion and stiffness virtual fixture-based controller. A dedicated 3 DOFs RCM platform is used for TSS hand global positioning. Design, modeling, and control of the TSS hand are discussed in detail with dedicated experimental validations. A sampling test based on human tele-operation is processed on the oral cavity model with excellent success rate. The proposed TOOS robot demonstrates a highly promising solution for tele-operated, safe, cost-effective, and quick deployable COVID-19 OP swab sampling.